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Abstract 



The accuracy of the feeling of knowing (FOK) was assessed with regard 
to recall and recognition under three conditions: advanced or no advanced 

organizers; learned or non-learned information; and, sex differences. Twenty 
subjects learned paired-associates and were tested for. recall and- recognition 
accompanied by ratings of FOK strength. The feeling— of -no t-knowing was more 
accurate for females under the non-learned condition. 
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AN ASSESSMENT OF THE FEELING-OF-RNOWING 



AND FACTORS INFLUENCING' ITS ACCURACY 
William E. ' Hauck , J. William Moore 
Bucknell University 
and Richard Isakson 
Cornell University 

The learner often experiences the situation in which he feels that he 
knows the correct response to a question even though he is unable, at the 
moment, to retrieve the necessary information from memory to provide an an- 
swer. The feeling accompanying the experience is referred to frequently as 
the feeling-of -knowing (FOR) » The FOR Is experienced countless times as the 
h uma n learner struggles to retrieve information stored daily in his memory 
system and has become the topic of recent research in information processing. 
Awareness of the accuracy of such feelings is important to the learning pro- 
cess since the degree to which the learner feels he. knows a designated body 
of knowledge may determine to a great extent whether he makes further efforts 
to learn. The general purpose of the present research was to determine the 
accuracy of feelings of knowing and not knowing, and to identify variables 
which may influence the accuracy. 

The role that the FOR plays in Information processing has been the subject 
of recent research by Hart (1965, 1966, 1967a, 1967b) who sought to determine 
the accuracy of the FOR. To establish conditions in which the FOR is experi- 
enced, Hart asked his subjects to answer general information questions (or to 
supply the response words in a paired -associate task); to give to each answer 
a rating of how strongly they felt they could recognize the correct answer even 
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if they had been unable to recall it; and, finally, to attempt to recognize 
the correct answers to the same questions presented, -in multiple choice form, 

Tc determine the accuracy of the FOK, Ss T performance on. the multiple choice 
questions was compared to their predicted ability to correctly recogaize an- 
swers. More specifically, feeling-of— knowing hits (FK) were compared to feel- 
ing-of-not knowing misses (FK) . A FK hit was scored when an j3_felt that he 
could correctly recognize an answer and then in fact did. currently, recognize 
it. A FK miss was scored when a subject rated a question as being one that 
he would not be able to correctly recognize, but then did correctly recognize. 
Hart * s analysis was thus based on those items which were correctly answered on 
the recognition test. FK misses and FK hits were not included in the assess- 
ment of the FOK since they represented items which the Ss had missed on the 
recognition test. Hart found significantly greater proportions of FK hits 
than FK misses. He concluded that the FOK is a "relatively accurate" indica- 
tor of what has been stored in memory. His assessment of the FOK was, how- 
ever, limited in the sense that only FK hits were compared .to . FK misses . Thus , 
the accuracy of the FK was relative only to the inaccuracy of the FK. Also, 
because the storage of information was not controlled. Hart had no way of know- 
ing if the items missed on the recognition test were the result of an inaccu- 
racy of the FK. Also, because the storage, of information was not controlled. 
Hart had no way of knowing if the items missed on* the recognition test, were 
the result of an inaccuracy in the Sis 1 retrieval process or the result of a 
lack of information storage. 

The same problems occurred in two other studies which used Hart's methods 
of analysis. Freedman and Laudauer (1966) found that Ss correctly recognized 
73% of the items they felt they knew, and were able to recognize 35% of the 
items they did not know. Hauck and Isakson (1971) also found significant 
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differences between proportions of FK hits C.55) and "FK misses (.29) using 
general information questions. 

A specific purpose of the present study was to overcome some of the prob- 
lems of previous research by controlling the storage of information so that 
direct comparisons could be made of the FOK on stored and non-stored informa- 
tion. Also, by controlling the degree of information storage, it is possible 
to assess the accuracy of the feeling-of-not-knowing (FK) . If £s can discrim- 
inate between learned and non-learned information and give FK ratings to the 
non- learned information, then the FK is accurate. Such a measure of the FK 
was not possible in previous studies. The FK is important to the efficiency 
of learning because if a person is not accurate in determining when informa- 
tion has not been stored in memory, he may waste a great amount of time trying 
to retrieve knowledge that was never stored. 

The accuracy of the FOK after recognition, or the accuracy of the con- 
fidence placed by a learner in a recognition has not been examined. The accu- 
racy of this second kind of FOK judgement is important to establish because 
the FOK judgement after recognition could determine whether or not a learner 
retains his responses or reconsiders and chooses others. A second purpose of 
the present research was to establish the accuracy of the FOK after recognition. 

Previous studies have not dealt with variables which may influence the 
accuracy of the FOK. Although it has been established that retrieval of infor- 
mation can be enhanced through the use of categorization or advanced organiza- 
tion of information being learned (Ausubel, 1960; Handler, 1967; Hiller, 1956, 
1967) , no research relates the FOK to organization. A thiru and final purpose 

of the present study was to determine if organization of information and sex 
differences influence the accuracy of the FOK. 
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In summary, the present research, attempted to answer the following four- 
part question. How accurate is the FOK. with regard to: (a) the degree of in- 

formation- .strrage; (b) recognition; (c) advanced organization of information; 
and, (d) sex differences? 

Method 

Design 

The general design of the experiment consisted of the recall- judgement- 
recognition paradigm employed by Hart, plus a second judgement by Ss after a 
recognition attempt. Various factorial designs were used with several depend- 
ent variables to test the effects of degree of information storage, sex, and 
mode of information storage (advanced organizers) on FOK accuracy. Each de- 
sign is indicated with the results of each analysis . 

Sub j ects 

Ten adult, males and 10 adult females of varying professional and educa- 
tion backgrounds with ages ranging from 20 to 45 years were randomly assigned 
to two treatment groups: advanced organizers and non-advanced organizers. 

The Ss were stratified according to sex with five males and five females in 
each group. 

Materials 

The materials consisted of : (a) paired-associates memo:.'ized by each S_i 

(b) three tests, one pretest of ability to memorize pa ire d - ass o ciates , and two 
posttests, one for recall and one for recognition of learned paired-associates ; 
and, (c) Likert type scales on which each _S rated his feeling-of-knowing to 
items on the post tests. 

The pretest was constructed from 20 randomly paired words, chosen for 
their- high frequency of. occurrence (at least 50 per million) as indicated by 

by the Thoradike-Lorge (1944) word count . Each item of the pretest was written 
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on a 3 X 5 card and consisted of the stimulus word of a word pair. Knowledge 
of results was presented for each Item on a 3 X 5 card, also. 
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Each of the two posttests consisted of 50 paired-associate items made up 
of stimulus words of a word pair which could be placed into one of five categories, 
In each category the stimulus word was linked to its response by a rule or con- 
cept. The concepts which served as labels for the five categories and an example 
of a word pair for each category are: 

1. Covers: the stimulus covers the response. 

Example: Tent - Cake 

2. Supports: the stimulus supports the response. 

Example: Table - Key 

the stimulus goes around the response. 

Chain - Boot 

the stimulus smashes the response. 

Brick - Gourd 

the stimulus goes, .into the response. 

Tire - Box. 

Ten paired-associates were grouped under each category.' Five pairs in 
each category were randomly selected for Ss to memorize. Thus when Ss responded 
to the posttests, 25 pairs of the 50 items had been memorized and the other 25 
fictitious. The arrangement of m emori red~~and fictitious items.. permitted the 
examination of responses to information which had been either stored .or not 
stored, end the five categories permitted . the study of advanced organizers. 

Each posttest consisted cf the same paired-associate words. The post test 
of re ca ll required the responder' to supply the. missing response word. The 
recognition test was multiple choice with four possible choices for answers. 

The three distractors chosen for each item were logical responses with respect 



3. Around: 
Example: 

4. Smashes: 
Example: 

5. Into: 
Example: 
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to the category of the correct choice. 

The Ss made their recall and recognition responses on answer sheets which 
were provided. A scale on which Ss could rate for FOK appeared adjacent to 
each item. The Ss rated their FOK on the following scale: 

YES NO 

6 5 4 3 2 1 

The following key to the FOK scale appeared at the top of each page of the 
answer sheets: 

6 5 4 3 2 1 

Definitely Fairly conf i- Slightly cer- Slightly cer- Fairly con- Definitely 
Know dent of know— tain of know- tain of not fident of do not know 

ing ing knowing not knowing 

Procedure 

In the first phase of the experiment , the pretest of paired-associate 
learning ability was administered individually to the Ss. The 10 PA’ s were 
first presented one at a time on 3 X 5 cards for approximately three seconds 
each. After the initial exposure each stimulus word was presented and Ss at- 
tempted to verbally supply the response word. Knowledge of results was given 
after each attempt. In the event of an incorrect response, S_ was allowed to 
see the entire PA again for three seconds. The pretest was completed as soon 
as S_ could go through all 10 pairs without making .any errors . The number of 
trials each jS needed to reach the criterion was recorded and this measure was 
used as a covariate in the analysis of covariance which was performed to test 
several of- the hypotheses of the study. 

One week after the pretest was adgrf nisteredy training began- on another 
group of PA’s on which Ss’ FOK and retention were to be tested. The Ss in the 
advanced organizer group (Treatment I) were instructed that they were to learn 
25 PA T s which had been grouped into five categories with five PA f s per category. 
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A sample pair, CAR - GARAGE, from the category "Into" was shown to the Ss. 

They were instructed to think of the category concepts as they were, shewn the 
PA’s during an initial exposure. The concept for each category was explained 
before exposure to the PA's in the category, began. When the instructions had 
been given, exposure to the PA's began. At the beginning of each category, 
an example of a PA fitting in the category was presented. Then, Ss were al- 
lowed to see each PA in the category- for three seconds. Subjects were exposed 
to and trained on one category at a time. 

The initial exposure for the non-advanced organizer group (Treatment II) 
was the same as that for Treatment X except Ss in the Treatment IX were told 
nothing about the categorization of the PA's.. Each S_ was shown the sample pair, 
CAR..- GARAGE, but was not given any directions as. to how to go about . learning 
the PA's. 

The order of presentation of the PA.' a within categories was the same for 
Ss in both treatment groups. Immediately following the initial exposure of the 
PA's within a category, the training trials were begun.- The .S saw. a stimulus 
word and tried to supply -the response word. After the attempt, J5 saw a card 
hearing both words of the PA, not: one of the original five PA's of the category, 
was inserted. The inserted pair fit logically into the category of the re- 
placed PA, but in no way entered into any subsequent test or analysis. They 
served the purpose of e l i m in a t i ng .re p e t i. tion of already learned pairs while 
maintaining a constant number, of pairs which Ss. were trying to learn at any one 
time. The order of the PA's within a category was varied over the learning trial 
When Ss gave the correct response for each of the original five, stimulus words in 
a category , training, ceased von. that: category and another was begun. The training 
continued until each S_ came to a 100% criterion on each .of. the five categories. 

One. week following training, Ss were tested individual ly on their recall 
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and recognition of the 25 PA T s they had learned during the training one week 
earlier. They were also tested on the* 25* non*>learned PA 1 s . After the pre- 
sentation of each stimulus word, £ attempted to recall the word that had been 
paired with it and to' write that - word- on the answer sheet. For each PA pre- 
sented made a FOK judgementbased' on how strongly he f elt that~ he; knew the 
response word and would be able' to recognize it- in the form of a multiple 
choice question. The rated his FOK; .on the six-point scale. This rating 
following' an attempt at recall constituted FOK Rating 1. The 50 PA's were pre- 
sented to 6 for recall and Rating 1 and then the recognition test; began. No 
knowledge of results on . the recall test was given. The stimulus words were 
again presented , but this time: they were' accompanied by four possible response 
words. The 8s chose the word they felt the stimulus. .word, had been paired with 
and recorded their answer.- After* each* recognition attempt , _Ss rated their FOK 
on the basis of how strongly they felt the word* they : had chosen was the correct 
response word. ' Once again this rating-, :F0K Rating 2, was made on the six-point 
scale provided on the answer sheet. The experiment was completed when all 20 
Ss had been individually tested for recall and recognition and had given their 
FOK judgements on Ratings 1 and 2. 

Results 

The mean proportions of FK and: FK~ hits and misses- for both FOK ratings , 
after, recall and' after recognition -, r on the combined-learned and non-learned 
pairs approximate most closely the data obtained in the studies by Hart (1965, 
1966, 1967a, 1967b) where degree; of- inf ormation storage' was not controlled. In 
comparing proportions of FK hits to FK: misses on Rating X (after recall), there, 
was no significant difference found: between the two proportions (.66 for FK hits 
and .41 for FK misses; t * 1.58, p>. 05, two-tailed). On Rating 2, which came 
after the recognition attempt* there was a significant difference found between 
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proportions of FK bits (.74) and FK misses (.30) (t — 3.31, p<.01). Thus for 
the combined learned and non— learned PA f s the findings of Hart were replicated 
only on the rating following recognition. 

In comparing proportions of FK hits: (.79) to FK misses (.65), it was found 
that there was no significant difference for Rating 1 (t = 1.04, p>.05, two- 
tailed). On Rating 2, however, there were significantly more FK hits (.87) 
than FK misses (.36), (t = 3.45, £<.01, two-tailed). Once again, as with the 
combined PA T s, the FOK judgements were accurate, as accuracy is defined by Hart, 
only on the rating following recognition and not on the recall rating on which 
Hart had found FOK accuracy. 

When both types of hits (FK and FK) were combined and compared to misses 
(FK and FK) on the 25 learned pairs there was no: .significant difference between 
Overall hits (.61) and. Overall misses (.39) -on Rating 1 (t = 1.00, £>.05, two- 
tailed). On Rating 2 there was a significant difference between proportions of 
Overall hits (.83) and Overall misses (. 17) , (t = 3. 95,- £<.05, two tailed). 

The-question of whether the factors, degree of - information storage (learned 
or non-learned) , sex of learner, and mode- of inf ormation storage (advanced and 
no advanced organizers) , have any: effects on the accuracy of- the FOK- was tested 
with the ANC0VA for Ratings 1 and 2 using four different dependent variables: 
Overall hits, FK hits, FK hits, and FK ratings. In all of the ANCQVA's, trials 
to criterion on the pretest served as the covariate. 

For Overall hits on Rating . 1, a three-way ANC0VA for repeated measures with 
sex, mode of storage, and degree of. storage as dependent variables showed no 
significant effects or interactions among- the variables. On Rating 2 there was 
a significant effect from degree of storage (F = 33.95, £<. 01) with more Overall 
hits achieved on the learned than on- the non-learned PA f -s. There were no other 
significant differences or interactions with Overall hits on Rating 2 as the 
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dependent-variable. 

With FK hits as the dependent variable* the. effects of sex, mode -of storage 
and degree of storage on EOK accuracy were tested. - On Rating l a three-way 
ANOVA indicated a significant effect- from: degree of st or age with more FK hits 
achieved on the non-learned PA's:(F * 55-. 68, jx<.01). The factors of sex and 
node of: information storage had" no significant: effects- on FK hits on Rating 1. 

For FK hits on Rating 2, the ANCOVA showed- significant effects resulting 
from -the factors' of mode of : storage (F** 5.40, £< .05) and degree of storage 
(F * 169. 00, £< . 001) . The' Ss: in the non-advanced organizer, group achieved more 
FK hits -than* did Ss in' the advanced: organizer group. With regard to degree of 
storage, the non-learned PA’s* produced- more FK hits* than did .the learned PA* s 
as was the case for* Rating .1. * The factor ' of- sex of learner had- no- effect on the 
dependent- variable^ There* was ," however*,' a significant interaction (F ■ 4.84, 

£<.05) between sexes with regard to* the non-leamed FA Vs . Females scored more 
FK hits than did males. 

A two-way ANCOVA with FK hits on the 25 learned PA’s as -the dependent 
variable and sex and mode of storage* as* independent- variables showed no significant 
effects* or* interactions on -either* Rating: 1' or: Rating 2. 

.. A two-way ANCOVA with FK ratings: on the 25' non-learhed PA’s as the dependent 
variable- and sex and. mode of: -storage .as- independent- variables showed no significant 
effects or interactions' on either: Rating* X' or Rating 2. 

To determine if the* advanced organizers used, in the experiment had a bene- 
ficial effect' on Ss ’ recognition' performance a. .two-way. ANCOVA was performed with 
correct recognition responses .as* the dependent variable and- sex and mode of in- 
formation storage as independent variables.. A- significant effect was produced by 
the mode -of storage factor with Ss* in the* advan c ed: orga n iz e r group correctly 
recognizing more* of the PA’ s than Ss in the non-advanced organizer group-. (F- .» 7.64, 
"<.05) . 
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Discussion 

The results of this -study: -contribute- to the information. concerning the 
acc ur ac y of- the~ f esling-of ^knowing: and' -the* effects* that* certain factors have 
on the ability to accurately assess what information one has- or has not stored 
in memory. 

In . general, the results' obtained . by.' Hart (1965, 1966 , 1967a, 1967b) in 
studying the' FOK were not replicated*. That .is, in arriving at a measure of FOK 
accuracy, the: proportions of ' FK: hits were* not significantly greater than the pro- 
portions of' FK misses for the' FOK* rating:. following recall. The FK and FK. hit 
and miss proportions on -the 25 learned and -the. 25 non-leamed PA's were combined 
in the present study to- a ppr o ximat e the^si t ua t ion: created by Hart where degree 
of information storage~:was~ not* controlled. The FK hit (.66) and FR miss (.41) 
proportions for Rating 1 .were alaaostr identical to those reported by the Hart 
(1967a) experiment. Statistically, the non-significance-could arise only if the 
within-group variance for the present study- -were greater than that for Hart's 
study. Thus the most- p lansi ble expl anation for the' result lies- in the differ- 
ence between' populatioas studied in Hart' s experiment and' the present experiment . 
In the present study, adults who: exhibited a wide' range of educational and in- 
tellectual differences served as Ss; however, Hart used college undergraduates 
who probably formed a more homogeneous: sample with respect to learning variables. 
Support for- the: explanation is found in the- Hauck and Isakson (1971) experiment in 
which Hart's findings were replicated' when college students were used as Ss. The 
identif ication within each population,' college student and ijon-student adults , of 
the exact variables which* lead' to* a' difference in information processing were not 
identified. 
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The FOK rating after recall. Rating 1, did not yield accurate judgements 
when FK hits and FK misses were compared for only the 25 learned PA's. Also 
when Overall (FK and FK) hits were compared to Overall misses on the learned 
PA's there were not significantly more hits than, misses. It can, therefore, be 
concluded that the FOK is not accurate among adult, non-student Ss for the FOK 
rating following recall. 

The FOK rating following recognition was found* to be more accurate than 
the recall rating. What is ft about FOK Rating 1. that: brings about lower FK 
hits proportion and higher FK miss proportions than, is seen on Rating 2? Aside 
from the fact that the added informational input from the multiple choice ques- 
tions makes Rating 2 somewhat less difficult, are there any other factors which 
might contribute to differences of accuracy? A possible explanation lies in the 
notion of response bias. Underwood C1966) states: "The fact is that we have 

response biases of a wide variety, and whenever we must make a decision in an 
ambiguous situation, these biases are likely to be involved in the decision. 

Even if the jS is responding with 'yes' or 'no* and has to make a decision on an 
ambiguous, threshold-measurement, trial, we may find him saying 'yes' more than 
*no*” Ipp. 186-1873. On the other hand, we may find the biased toward 
"no" more than "yes" and this: is what appears* to have happened in the present 
study. The Ss were observed to be worried about rating PA's, FK, and then not 
being able to recognize them correctly. Indeed, one did* not give a FK rating 
to a . single PA from among the , 25 he had* learned the: week before. Yet on the 
recognition test his performance was above average. This "better to be safe 
than sorry" bias in making FOK ratings after recall caused many of the learned 
PA's to be rated FK. On the average, 9.3 of the learned PA's were rated FK on 
Rating 1. These items resulted . in FK misses when they were subsequently recog-, 
nized correctly. The response bias was not as strong on Rating 2 where FK. ratings 
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on learned PA's averaged only 5.1. The underlying reason for the response bias 
toward hesitancy in giving FK ratings is a personality variable which remains to 
be explored. 

The results of the present study shed some light on the question of what 
factors influence the accuracy of the FOK. Degree of information storage was 
found to have the effect that would be expected . The Ss achieved significantly 
more Overall hits than misses on Rating 2 on ; the learned PA's. In general it 
would seem that Ss are able to make more accurate judgements about information 
that has been stored in memory thair about information that has not been stored. 
The nature of the recognition task, however, may have led to a response bias 
where Ss rated many of the non-learned PA's FK simply because of the confidence 
they felt from recognizing the learned PA's and giving them FK ratings. The 
fact that there were no differences in number of Overall hits between learned 
and non- learned PA's on Rating 1 would tend to support the explanation of a re- 
sponse bias. On Rating. 1 the recall task was sufficiently difficult so that Ss 
did not feel overly confident in their ability to correctly recognize the re- 
sponse words. In fact, the bias seemed to be toward being too cautious in. giv- 
ing FK ratings. 

The factor, degree of storage, also affected the number of FK hits. More 
FK hits were achieved on the non-leamed PA's than on the learned PA's. This 
result was expected since Ss rated -significantly more of. the non-leamed pairs • 

FK than FK. The fact that these FK ratings on the non-leamed PA's resulted in 
FK hits is natural since the Ss were expected’ not to correctly recognize the re- 
sponse words for non-leamed PA's. Degree of storage, then, was found to indicate 
that Ss were generally able to discriminate between the learned, and non-leamed 
information. 

The factor , sex of learner , had no effect on FOK accuracy except in the 
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instance of FK hits on Rating 2 where it was found that female Ss scored more 
FK hits on the non-learned PA's than did males. An explanation of this result 
must await further research into sex differences^ perhaps in the area of differ- 
ential response styles. 

It is of interest to note that the factor , mode of information storage, 
had little or no effect on FOK accuracy especially in light of the fact that 
the advanced organizers were found to facilitate correct recognition of the 
response words. The only instance where mode of storage influenced FOK accuracy 
was in the achieving of FK hits on Rating 2 where Ss in the non-advanced or- 
ganizer group out-performed Ss who had' been trained with advanced organizers. 

This effect could be partially due to Ss in the advanced organizer group rating 
fewer items FK than did Ss in the non-advanced organizer group. Thus, the ad- 
vanced organizer group got fewer FK hits because they were able to correctly, 
recognize more of the PA's than did the Ss using no advanced: organizers. Fewer 
FK ratings and more correct recognitions would then cause the Ss using advanced 
organizers to achieve fewer FK hits. If advanced organizers facilitate recog- 
nition performance, as they were found to do, why is the FOK accuracy not facil- 
tated correspondingly? Subjects possibly do not realize that advanced orga- 
nizers can have a facili tat ive effect on recognition and' thus they respond as if 
no advanced organization were present. A descrepancy between FOK judgement and 
recognition performance could result and the FOK accuracy decrease. 

When the. accuracy level of the FOK has. been firmly established, the FOK 
rating could be used in studies of human learning to determine what the learner 
has stored in memory even though' the information may not be accessible to im- 
mediate retrieval. If the human learner had a retrieval system that allowed him 
to determine exactly what had been stored in memory and to bring that information 
to a conscious awareness, there would be no need to study the FOK. This, however,. 
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is not the case. Since the accurate retrieval of information plays a vital 
role in the student’s success in school, the research done to find ways of 
improving the efficiency cf assessing' what Information has or has not been 
stored in memory has direct application in the educational process. 
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